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Abstract 

This study focused on college students’ development of conceptual knowledge in geographic information 
system (GIS). The aim of this study was to examine if and how students developed their conceptual 
knowledge during their enrollment in an introductory-level GIS course. Twelve undergraduate students 
constructed 36 concept maps and revised 24 concept maps in three sessions. The author scored those 60 
concept maps in two different ways. The first method measured the degree to which concept maps were 
structurally complex; the second method assessed the correctness of interrelationships between spatial 
concepts shown in the maps. A statistical analysis of the scores obtained through the second method 
suggested that there was a significant difference between the maps created in the first session and the maps 
revised in the second session. Students could successfully revise their original concept maps at the middle 
of a semester. A mix of the two quantitative and qualitative methods had the potential to examine the 
development of students’ conceptual knowledge through multiple perspectives. Lastly, this study discusses 
how concept maps can be applied to research and instructions. Concept maps can be used for exploring 
students’ understanding of spatial concepts. 
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The purpose of this paper is to make two, related arguments. First, that it is essential 
that Geographic Information System (GIS) educators pay attention to, and explicitly 
teach the geospatial concepts that are the building blocks and language of GIS and 
second, that concept maps can be used to analyze students’ conceptual knowledge. To 
begin, the author briefly discusses the role of geospatial concepts in GIS education. A 
review of previous research using concept maps follows. Then this paper reports on new 
research using concept maps focused on whether college students can improve their 
conceptual knowledge about spatial thinking while taking a GIS course. The paper 
concludes with a discussion of implications for GIS educators. 

Background 

When it comes to the effects of computer technology on students’ learning, the 
discussion is varied among education researchers. Computer-based media provide more 
interactive communication than traditional media; as a result, students can understand 
difficult concepts easily and solve problems (Guerrero, Walker, & Dugdale, 2004; 
National Research Council, 2000). On the other hand, some educators anticipate that 
students tend to learn how to use computer technology and neglect fundamental 
concepts necessary for problem solving (Barak, 2004; Becker, 1993). 

This dichotomy can be seen in GIS education. GIS is supposed to be a useful support 
system for spatial problem solving and decision making (Baker & White, 2003; 
National Research Council, 2006; Shin, 2006), GIS education, especially in 
introductory-level college courses. However, it tends to emphasize GIS operational and 
marketable skills and teach about GIS rather than with it (Brown, Elmes, Kemp, Macey, 
& Mark, 2003; Sui, 1995). In an unfavorable case, some GIS novices are overwhelmed 
by a series of GIS operations in a tutorial. They follow a GIS manual without thinking 
of what questions might be possible, why a step of an operation was necessary, and 
what outcomes were available. Even if they successfully obtain full credit for a tutorial, 
they would neglect the related fundamental concepts that are necessary for reading and 
interpreting maps and creating maps and spatial hypotheses. Worse yet, some students 
may complete a GIS course without attaining spatial thinking skills required for 
problem solving, information retrieval, and critical thinking (Thompson, 1991). As 
some people blindly trust outcomes brought about by calculators, some students are 
more likely to believe outputs brought about by GIS software without assessing them 
critically. Ironically, GIS software as a tool for assisting users’ spatial thinking tends to 
become a substitute for the act of thinking spatially (Downs, 1997). 

The advances in real-world GIS data availability, software and hardware usability, 
and geospatial technology accessibility has attracted a wide range of users and provided 
them with valuable outputs through spatial problem solving. As a consequence, people 
recognize that GIS is an effective tool for reinforcing students’ spatial thinking 
competency (Goodchild, 2006; Kerski, 2008; National Research Council, 2006). Even 
though conceptual knowledge supports higher-order thinking such as problem solving 
(Howard, 1987), there is little empirical research on college students’ conceptual 
knowledge in geospatial science (Baker et al., 2015). This study examined if university 
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students developed their knowledge related to spatial concepts during an introductory- 
level GIS course. 

Geospatial Concepts in GIS Education 

Concepts denote the regularities of objects or events (Novak & Go win, 1984) by 
generalizing multiple instances based on common critical attributes (Howard, 1987; 
Smith & Medin, 1981). For example, the concept of bird is a label that conveys some 
common characteristics found among the related species; the concept reminds people of 
the vertebrates that lay eggs and possess a beak, feathers, and wings typically for flying. 
As a result, concepts enable individuals to mentally separate examples from non¬ 
examples (Klausmeier, 1992) and utilize the concepts for communication or higher- 
order thinking such as problem solving (Ausubel, Novak, & Hanesian, 1986; Howard, 
1987; Medin, Lynch, & Solomon, 2000). When it comes to spatial phenomena and 
relationships that can be observed at a mesoscale, a variety of concepts have been 
coined and used by geographers (DeMers, 2009). For example, cluster, density, 
diffusion, dispersion, distribution, and pattern are concepts for understanding and 
communicating regularities about spatially arranged geographic features. Concepts 
denoting geographic phenomena are called geospatial concepts (Golledge, Marsh, & 
Battersby, 2008). 

There are three reasons why geospatial concepts should explicitly be taught in 
educational settings. First, geospatial concepts are one of the three elements that support 
people’s spatial thinking. Geospatial concepts are intertwined with the other two 
elements of spatial thinking, which are representations and processes of reasoning, 
essential for decision making and problem solving related to spatial entities for various 
purposes, contexts, and disciplines (National Research Council, 2006). The second 
reason is that geospatial concepts are intensively and frequently used for obtaining two 
types of geospatial information through maps. First, geospatial concepts are used to 
interpret map features. For example, people identify locations by considering map scale 
(Liben & Downs, 1991), understanding spatial layouts or relationships (Uttal, 2000) and 
calculating distances, areas, and directions (Crosby, 1997). Second, geospatial concepts 
are used to understand geospatial information that is not explicitly represented in a map. 
Some information can be obtained only through map readers’ spatial inference or 
reasoning. For instance, map readers uncover new relationships by overlaying multiple 
map features (Battersby, Golledge, & Marsh, 2006). 

A third reason for introducing geospatial concepts explicitly in GIS education is 
some geospatial concepts essential for successful GIS use are not easily or casually 
learned through experience and should be taught systematically in a formal setting. 
Some concepts are simple enough to understand and work with. In contrast, other 
concepts are complicated and are compounds of simpler concepts. The ontology of 
geospatial concepts in terms of complexity has been studied by several researchers 
(Golledge et al., 2008; Golledge, 1995; Kaufman, 2004; Marsh, Golledge, & Battersby, 
2007; Nystuen, 1968; Papageorgiou, 1969). Although the complexity levels of 
geospatial concepts are still a matter of discussion and research, Golledge et al. (2008) 
classified geospatial concepts into five levels and identified complicated or complex 
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concepts such as association, buffer, distortion, scale, and projection. These concepts 
are essential for basic map interpretation and spatial analysis. However, students are not 
likely to understand those concepts fully and meaningfully unless they are taught 
explicitly. 

Thus, geospatial concepts have a foundational role in GIS education. However, there 
is little empirical research on college students’ conceptual knowledge in geospatial 
science. One note of caution: students do not come to the classroom tabula rasa. 
Geospatial concepts can be attained not only in the classroom but also in daily life. 
Students utilize their personal, informal conceptual knowledge, whether it complies 
with accepted and accurate understandings, for further knowledge development in 
formal settings such as the classroom. When students learn new concepts, they have to 
resolve and connect them with the concepts that they have already learned. Refining or 
restructuring conceptual knowledge occurs as a process of students’ conceptual 
development (Ausubel et al., 1986; Rumelhart & Norman, 1978). Understanding what 
concepts students have attained so far and how they develop conceptual knowledge 
would give instructors insightful feedback for teaching. 

Concept Maps for Assessment 

Concept maps have been used by educators (Novak & Go win, 1984) to evaluate 
students’ conceptual knowledge in a range of subjects. For example, they were biology 
(Barenholz & Tamir, 1992; Jegede, Alaiyemola, & Okebukola, 1990; Martin, Mintzes, 
& Clavijo, 2000; Wallace & Mintzes, 1990), chemistry (Ross & Munby, 1991; 
Schreiber & Abegg, 1991; Wilson, 1994), physics (Roth & Roychoudhury, 1993), 
medical science (Mahler, Hoz, Fischl, Tov-Ly, & Lernau, 1991), statistics (Roberts, 
1999), education (Beyerbach, 1988; Lay-Dopyera & Beyerbach, 1983), history (Herl & 
Baker, 1996) and geography (Oda, 2011; Rebich & Gautier, 2005; Walshe, 2008). 

One of the reasons why concept maps have been used for research and assessment is 
that these constructions make visible how people connect and understand the concepts 
that they have learned. The main components of concept maps are concept nodes and 
labeled links (Figure 1). A concept node is connected to another concept node by a 
labeled link. This node-link-node composes a proposition, which is the minimum unit 
that can represent the meaning of a concept. Also, a crosslink is sometimes placed 
between two concept nodes at the lower hierarchy level. Sets of propositions are 
arranged in a network form to convey the ideas of the primary concept located at the top 
of a map. To construct a concept map, people recall concepts related to a primary 
concept, consider the attributes and relationships of these terms and arrange them 
meaningfully (Jonassen, Reeves, Hong, Harvey, & Peters, 1997; Wandersee, 1990). As 
a consequence, concept maps can effectively show how people cognitively interrelate 
concepts, and in a more efficient fashion for researchers than other assessment methods 
(Jonassen, Beissner, & Yacci, 1993; Ruiz-Primo & Shavelson, 1996; White & 
Gunstone, 1992). Although clinical interviews can also reveal students’ understandings 
in depth (Walshe, 2008), a description in a linear fashion tends to be lengthy to convey 
a complex thought on relationships among multiple concepts. As for traditional 
assessment methods, multiple-choice exams usually require students to answer a 
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considerable number of questions with little metacognitive reflection (Novak, 1998; 
Rebich & Gautier, 2005). 



Figure 1. The components of a concept map 


Two methods have typically been used to analyze concept maps. The first method 
(structural) is to count the number of map components such as concept nodes, links, 
propositions, and crosslinks (Novak & Gowin, 1984). For the Novak and Gowin (1984) 
scoring method, each level of hierarchy receives five points; each valid crosslink 
receives ten points. In contrast, both a valid proposition and an example receive only 
one point each. A valid crosslink is regarded as a good indicator of integration of 
multiple concepts (Novak & Gowin, 1984). Furthermore, this method has been 
modified by other researchers (Markham, Mintzes, & Jones, 1994; Stuart, 1985). In 
short, the structural scoring scheme emphasizes the complexity of hierarchical network 
forms reflecting well developed conceptual knowledge. 

The second method (relational) places greater value on propositions than the first 
method. The fundamental assumption is that a proposition is a minimum unit of the 
meaning that can be judged regarding the validity of an interrelationship between two 
concepts (Ruiz-Primo, Schultz, & Shavelson, 1997). There are variations of scoring 
methods. Rice, Ryan, and Samson (1998) compared three types of proposition-based 
scoring methods. These three categories have the following different scoring criteria: (1) 
whether a pertinent concept exists, (2) whether a correct relationship between pertinent 
concepts exists; and (3) whether an incorrect relationship exists. Rice et al. (1998) 
concluded that the second criterion is the most useful as a class assessment because the 
scores obtained with the criteria correlated with scores on related multiple choice tests. 
Roberts (1999) modified the conventional scoring scheme by weighing the accuracy of 
propositions. There were several reasons why this modification occurred. Some concept 
maps included incorrect propositional links and links with no words. In this case, 
counting the number of map components would not work for incorrect propositions. 
Moreover, the concepts that could be used in concept mapping were assigned to 
subjects in advance. This methodological aspect made counting the number of concepts 
and propositions less significant. Ruiz-Primo et al. (1997) used a square matrix and a 
propositional inventory to score their concept maps by focusing on the quality of 
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propositions. The matrix included all possible pairs between concepts; the propositional 
inventory was used to evaluate the variation in the quality of proposition with a five- 
level scale: valid excellent, valid good, valid poor, don’t care and invalid. Rye and 
Rubba (2002) utilized expert maps in their concept map scoring. For the assessment of 
propositions, they examined the degree to which a novice’s and an expert’s map 
matched. The latter two studies adopted master models such as a propositional 
inventory and an expert map. 

To summarize, students use geospatial concepts intensively in activities involving 
geospatial technologies like GIS and learn new concepts in formal classroom settings 
through building on their existing conceptual knowledge. Although some researchers 
have reported the use of concept maps in geography (Rebich & Gautier, 2005; Walshe, 
2008), there is little empirical research on how educators assess students' conceptual 
knowledge in GIS. This study used concept maps as a tool to explore how students 
developed their conceptual knowledge and analyzed undergraduate students’ geospatial 
concept maps by adopting the two scoring schemes: the structural and the relational. 

Domain 

This study focused on undergraduate students who enrolled in either of two 
introductory-level GIS courses offered at a large state university in Texas. The two 
courses were “GEOG 390 Principles of GIS” offered by the Department of Geography 
and “FRSC 461 GIS for Resource Management” offered by the Department of 
Ecosystem Science and Management. The author observed the lectures and laboratories 
of the two GIS courses to investigate when students learn specific GIS topics in the 
classes and when they perform various GIS activities in laboratories. 

Although both GEOG 390 and FRSC 461 were introductory-level GIS courses, these 
courses differed from one another regarding philosophy. The fundamental stance of 
GEOG 390 was geography as the home discipline of GIS. The lectures and laboratory 
sessions in this course encouraged students to consider and understand the principles 
that exist behind knowledge and skills related to GIS. On the other hand, the 
fundamental stance of FRSC 461 was GIS as an enabling technology for science. This 
course emphasized scientific inquiry and applications in resource management through 
lectures about geospatial tools and a final project that involved problem solving. 
Although the two courses varied in philosophy, there was a commonality regarding GIS 
topic arrangements in the lectures and laboratory sessions (Tables 1 and 2). Both the 
courses above focused on GIS core competencies and knowledge and the cartographic 
aspects of GIS in the first half of the semester. In the latter half of the semester, both 
courses gradually moved to topics about GIS analysis. 

There were further differences in the configuration of the two laboratories. In GEOG 
390, each laboratory session had a focus topic and a mini project. In each mini project, 
students were required to apply the knowledge and skills of the topic. In FRSC 461, the 
laboratory had a final project. This project required students to choose a project topic 
with his or her partner by the fifth week of the course. In the final project, students were 
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required to solve an environmental question that did not have a correct answer and to 
deliver presentations about their final projects. 

Table 1 


The Lecture Topics Included in the Two Introductory-Level GIS Courses 


Week 

GEOG 390 

FRSC 461 

1 

GIS components. Data dimensionality. 

GIS definition, Spatial factors, GIS 

Measurements 

applications 

o 

Map design principles, Map scale, 

Components of GIS, Spatial data, 

Ai 

Coordinate system, Projection 

Attribute data, Cartographic model 

3 

Thematic map, Geoid, Ellipsoids, 
Datum 

Datum, Projection, Coordinate system 

4 

Projection, Datum, Coordinate system 

Projection, Coordinates, Datum 

5 

Map data entry 

ArcGIS file type, Enterprise GIS 

6 

GIS data structure, Topology 

Data sources, Data standards, GIS 
operations 

7 

GIS data types, Database management 

Metadata, Database management 


system 

system 

8 

Spatial analysis, Overlay, Boolean, 
Buffer, 

Grids, DEMs, TINs 

9 

Continuous data, Raster 

GPS 

10 

Raster, Interpolation 

Remote sensing, Raster analysis 

11 

Interpolation 

Remote sensing, Raster analysis 

12 

Raster analysis. Terrain analysis 

Remote sensing. Raster analysis 

13 

Raster modeling 

GPS Activity 

Table 2 

The Lab Activities Included in the Two Introductory-Level GIS Courses 

Week 

GEOG 390 

FRSC 461 

1 

Become familiar with ArcGIS 

Become familiar with ArcGIS 

2 

Become familiar with ArcMap 

Become familiar with ArcGIS 

3 

Create thematic maps and layout hard 

Manage projections and coordinate 


copy maps 

systems 

A 

Manage projections, coordinate 

Download online raster and vector 

e -r 

systems and metadata 

GIS data and manage meta data 

5 

Set georeferences 

Work with attribute tables and create 
maps 

6 

Work with attribute tables and queries 

Work with georeferencing, buffering 
and interpolation 

7 

Work with spatial queries and spatial 
joins 

Layout hard copy maps 

8 

Work with map overlay and spatial 
query 

Perform spatial analysis 

9 

Edit GIS map data 

Perform attribute query and attribute 
table processing 
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10 

Work with interpolation 

and Perform digitization, interpolation, and 

geo statistics 

spatial analysis 

11 

Perform raster analysis 

Final Project 

12 

Perform raster analysis 

Final Project 

13 

Final lab exam 

Final Project 


Methods 

This study analyzed college students’ concept maps to examine if and how they 
developed their conceptual knowledge during their enrollment in one or other of the two 
courses above. The methodology was a single-group time series design. Each subject 
attended a training session and three experiment sessions voluntarily in a single 
semester. The author recruited ten and seven undergraduate students through a flyer at 
the beginning of the 2008 fall semester and the 2009 spring semester respectively. Each 
participant received monetary compensation for each experiment session. Of the 
seventeen participants, four withdrew from the study. One participant constructed 
concept maps in a linear fashion rather than a hierarchical network form. Since the 
hierarchy was unclear, the author excluded his concept maps from the analysis. As a 
result, this study analyzed data provided by twelve participants (Table 3) through two 
different types of analysis: the structural and the relational. The twelve participants also 
answered about their map use and confidence in map reading. Their experience and 
confidence on map reading were relatively high. Five participants responded that they 
used a map about once a week; six participants used a map about once a month; only 
one participant used a map about once a half year. As for their confidence, two 
participants strongly agreed that they were good at reading a map; eight participants 
thought that they were good at reading a map; two participants neither agreed nor 
disagreed that they were good at reading a map. 

Table 3 


Distribution of Participants ’ Academic Characteristics 


Gender 

Male 

Female 


11 



1 


Age 

20 Years Old 


21 Years Old 

22 Years Old 


23 Years Old 

4 


4 

2 


2 

Academic Year 

Freshman 

0 


Sophomore 

2 

Junior 

5 


Senior 

5 



Spatial 

Sciences 

3 



Wildlife & 

Major 

Geography 

4 


Marketing 

1 


Fisheries 

Sciences 

A 

GIS Course 

GEOG390 

FRSC461 


7 



5 


Spatial Science 

GEOG390 or FRSC461 is the 

GEOG390 or FRSC461 is not the 

Course Work 

first spatial 

science course. 

first spatial 

science course. 

Experience 


7 



5 
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Procedures 

At the beginning of a semester, participants attended a 50-minute workshop and 
learned how to construct a concept map using specific software designed for that 
purpose. The contents and activities followed strategies introduced by Novak and 
Gowin (1984). In the training session, participants learned about concept maps in 
general and how to use the software, CmapTools, and then constructed a concept map 
about Earth by using concepts related to ecosystem and following step-by-step 
instructions. In the last part of the training session, they constructed a concept map 
about water without any instruction. The author checked their concept maps about water 
to see if participants properly understood concept mapping and how to use the software. 
Students participated in three sessions following this training session, roughly five 
weeks apart (Table 4). In the first session, participants created a concept map. In the 
second and third sessions, they created a new concept map and revised the map they 
constructed in the previous session. The average length of each of the concept mapping 
sessions was approximately 28 minutes. 

In each of the three sessions, participants started the mapping software to create a 
new concept map. The window had already included 30 geospatial concepts (Table 5) at 
the beginning of their mapping. Participants were asked to use only the concepts they 
were familiar with and arrange them to construct a concept map. Even though this study 
aimed at evaluating the development of students’ conceptual knowledge in a semester 
and necessitated multiple mapping sessions, students might have got used to mapping 
by using the software. The populated 30 geospatial concepts were relevant to the 
property of geospatial objects, analysis, and visualization. The professor who taught 
GEOG 390 and the author selected the 30 geospatial concepts to be used in the three 
experiment sessions by examining two chapters of a GIS textbook written by DeMers 
(2009). This textbook meant to introduce the GIS concepts and technologies that 
students should know in an early-stage of learning and used in GEOG 390. A set of the 
30 geospatial concepts may not be inclusive, though there were two rationales for the 
concept selection. The first was that these geospatial concepts should be covered in 
introductory-level GIS courses. The second was that the concept collection should 
engage a diversity of mental activities: aerial perception, spatial relationship 
representations, geometric manipulation, and spatial reasoning. The two chapters of the 
textbook were “Chapter 0: Spatial Lerner’s Permit” and “Chapter 2: Basic Geographic 
Concepts.” They focus on the geospatial concepts that underlie the nature of geospatial 
phenomena and spatial thinking and introduce the 30 geospatial concepts except overlay 
and topology. However, these concepts are relevant to some concepts introduced in 
Chapter 2. It describes how geographic entities such as points, lines, and polygons are 
represented and stored, and how multiple different types of geospatial entities associate 
or correlate each other. Moreover, the two concepts tie in with mental activities 
including spatial representations, geometric manipulation, and spatial reasoning. 
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Table 4 

Activities Conducted in the Three Sessions 


Step 

Activity 

1st Session 

2nd Session 

3rd Session 

1 

Answering question about subjects’ 
personal information 

X 



2 

Making a concept map about the 
primary concept, space 

X 

X 

X 

3 

Revising a concept map created in 
the previous session 


X 

X 

Table 5 





Thirty Geospatial Concepts Used 





Arrangements, Association, Boundary, Buffer, Cluster, Coordinate, Density, Diffusion, 
Direction, Dispersion, Distance, Distortion, Distribution, Line, Linkage, Location, Map 
projection, Network, Overlay, Pattern, Point, Polygon, Proximity, Scale, Shape, Size, 
Spatial relationship, Three dimensions, Two dimensions, Topology 


Assessment of Concept Maps 

The author obtained 36 original concept maps and 24 revised concept maps and 
scored them by utilizing the two scoring schemes: the structural and the relational. For 
the structural scheme, this study adopted a modified version of Novak and Gowin’s 
scoring method. This method was used by Markham et al. (1994) and counts the 
numbers of concepts, relationships, branchings, hierarchies, crosslinks and examples. In 
this study, participants used concepts assigned by the author in advance. Therefore, this 
study eliminated the count of examples. Each concept and each valid relationship 
received one point, respectively. Branchings had two scoring weights. The first 
branching received one point, and a successive branching received three points. Each 
hierarchy received five points. Each valid crosslink received 10 points because 
crosslinks can be regarded as evidence of concept map complexity. In each concept 
map, points were summed up to obtain a total score. 

The relational scheme evaluated each proposition shown in a concept map to decide 
a made statement was correct or not. For the decision, the author identified correct, 
partially correct, and incorrect concept pairs and correct propositional statements and 
use them as a reference. In the first step of this analysis, the author obtained experts’ 
definitions of the 30 geospatial concepts by examining two books and a GIS dictionary 
on the website of Esri, a leading GIS software vendor. This online dictionary included 
the terms of GIS technologies and cartography. One of the two books was a GIS 
textbook (DeMers, 2009) used for identifying the 30 geospatial concepts. The other 
book (Witthuhn, Brandt, & Demko, 1976) was referred to in a chapter on spatial 
concepts in the DeMers text and described core concepts in geography for students who 
had no previous exposure to geography. For example, an identified experts’ definition 
of cluster was “cluster demonstrates a type of distribution with a high density of 
features.” Hereafter, the concept of cluster will be used to explain an example of the 
following steps in the relational scheme. 


185 




Oda, K/ Concept Maps as a Tool to Analyse College Students' Knowledge of Geospatial.... 

The second major step examined each of the obtained definitions to consider which 
of the other 29 populated geospatial concepts would be candidates for the pairs that can 
align with experts’ definition. In the case of cluster, the candidates of concept pairs were 
distribution and density since the identified experts’ definition included the two 
concepts. The identified candidates were used to develop the concept pairs that would 
compose correct propositions. For example, a pair of cluster and distribution and a pair 
of cluster and density were made and assigned to a correct category. Then, the author 
formulated experts’ propositional statements by referring to the definitions and correct 
pairs. As an example, the statements about cluster are as follows: (1) cluster 
demonstrates a high density; (2) cluster demonstrates a type of distribution; (3) density 
is a measure of cluster; (4) distribution representing a convergent condition is a cluster. 
Establishing correct pairs and propositional statements enabled each of the 30 spatial 
concepts to have one or more correct pairs and two or more correct experts’ 
propositional statements. The author used them to decide if students’ propositions were 
correct or not. 

In the third major step, the author considered the possible pairs and propositional 
statements that may not belong to the correct category. For instance, the terms cluster 
and diffusion may bring the following statement: “cluster is one of the results of 
diffusion;” the relationship between cluster and spatial relationship can be expressed by 
the following statement: “cluster can be used to describe a spatial relationship.” These 
propositional statements are not consistent with the definition of cluster formulated by 
experts, nor are they overarching concepts; however, those statements are correct under 
certain circumstances. Thus, a combination of cluster and diffusion and a combination 
of cluster and spatial relationship are partially correct pairs and can formulate partially 
correct statements. After the author identified the pairs that belong to the correct and 
partially correct categories, he assigned the possible pairs that belonged to neither the 
correct nor the partially correct categories to an incorrect pair category. After all, all of 
the 435 possible pairs composed of the 30 geospatial concepts were categorized into 
correct, partially correct, or incorrect. The author transcribed these categorized pairs and 
the corresponding experts’ propositional statements into a propositional matrix for 
assessment. 

To initiate scoring with the relational scheme, the author rewrote all of the 
propositional pairs and the statements shown in each of the participants’ concept maps 
into a matrix. Then, he scored the pairs and statements based on the three categories of 
propositional pairs and statements. Since this scoring involved only one assessor, it 
might not have avoided subjectivity fully. The scores for propositional pairs ranged 
from 0 to 2 points. If a pair belonged to the correct category, the pair received 2 points; 
if a pair was a partially correct, the pair received 1 point; if a pair was neither correct 
nor partially correct, it did not get any points. The scores for propositional statements 
ranged from 0 to 4 points. A correct statement received 4 points; a partially correct 
statement received 2 points; an incorrect statement and a link without a statement did 
not receive any points. A combination of pair scores and statement scores established 
nine different accuracy categories (Table 6). Since a correct statement logically exists in 
only a correct pair, a score was not assigned to a partially correct or incorrect pair either. 
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Likewise, a score was not assigned to an incorrect pair because a partially correct 
statement logically cannot exist in an incorrect pair. As a consequence, the combined 
scores ranged from 0 to 6 points. If a participant’s proposition met the experts’ 
definition, the proposition received 6 points in total. 

Table 6 


The Relational Score Weight Matrix 



Correct Pair 

Partially Correct Pair 

Incorrect Pair 

Correct Statement 

6 points 

No assigned 

No assigned 

Partially Correct Statement 

4 points 

3 points 

No assigned 

Incorrect Statement 

2 points 

1 point 

0 point 

Missing Statement 

2 points 

1 point 

0 point 


Findings 

The author obtained the scores of concept maps constructed or revised by twelve 
students through three sessions. Then, he analyzed the scores through descriptive 
statistics and non-parametric statistics to examine if there were any differences among 
the sessions. Most of the students obtained their highest score when they created a new 
concept map in the second or third experiment session or revised their map in the 
second session (Tables 7 and 8). In the structural method, the largest number of 
participants obtained the highest scores when they created a new concept map in the 
third session. For the relational scores, the largest number of participants obtained the 
highest scores when they created a new concept map in the second session. Differences 
between high- and low-score maps were due to the number of map components and 
valid propositions. For example, Participant C (Figure 2) used 30 concepts in the third 
experiment session while Participant D (Figure 3) used 17 concepts in the first session. 
The largest factor contributing to the difference of map scores was the number of 
crosslinks. Participant C added six valid crosslinks; Participant D did not make any 
crosslinks. Since the score weight of crosslinks was larger than the other weights, the 
structural score difference was exaggerated. Differences between the two maps also 
affect their relational scores. The Participant C map included 35 correct or partially 
correct statements and 38 correct or partially correct pairs, while the Participant D map 
contained 14 correct or partially correct statements and 14 correct or partially correct 
pairs. 

The descriptive statistic of the individuals’ scores suggested some variations among 
sessions (Table 9). For the structural method, both the mean and median of the scores of 
concept maps newly created in the third session was the highest among all of the 
sessions. In contrast, both the mean and median of scores of maps newly developed in 
the second session was the lowest. Thus, the mean and median decreased in the second 
session and increased abruptly in the third session. The results obtained through the 
relational method were different from those through the structural method. Both the 
mean and median of scores of concept maps revised in the second session were the 
highest among all of the sessions. When it focused on only concept maps newly created 
in the three sessions, the mean of the relational scores gradually increased from the first 
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session to the third session. The descriptive statistic also provided different findings 
when the author compared the mean and median of the scores of maps newly created in 
the first and second sessions and revised in the second and third sessions respectively. 
In both of the structural and relational methods, the mean and median increased in all of 
the cases except the structural median of the scores of maps newly created in the first 
session and maps revised in the second session. Particularly, the mean and median of 
the relational scores between the new maps of the first session and the revised maps of 
the second session increased by 12 points and 12.5 points respectively. 

Table 7 


Each Participant’s Structural Scores 


Session 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

Created in Session 1 

92 

88 

99 

65 

89 

101 

78 

116 

96 

94 

93 

98 

Created in Session 2 

77 

73 

98 

76 

116 

87 

89 

105 

81 

89 

112 

72 

Revised in Session 2 

89 

91 

109 

90 

92 

87 

84 

140 

86 

85 

87 

98 

Created in Session 3 

96 

72 

164 

70 

97 

103 

72 

106 

111 

100 

98 

87 

Revised in Session 3 

96 

80 

99 

83 

115 

95 

85 

123 

87 

90 

115 

71 


Note: the scores written in bold are the individual’s highest score among the sessions. 


Table 8 


Each Participant’s Relational Scores 


Session 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

Created in Session 1 

54 

97 

89 

70 

93 

84 

40 

109 

109 

56 

66 

85 

Created in Session 2 

59 

67 

90 

84 

96 

102 

67 

67 

125 

62 

90 

88 

Revised in Session 2 

59 

94 

110 

117 

108 

102 

55 

123 

100 

62 

79 

87 

Created in Session 3 

52 

79 

136 

80 

114 

104 

64 

68 

106 

82 

81 

66 

Revised in Session 3 

52 

71 

98 

97 

96 

102 

66 

104 

118 

68 

76 

82 


Note: the scores written in bold are the individual’s highest score among the sessions. 


Table 9 

Mean and Median Values of Participants ’ Concept Map Scores 


Session 


Newly created 
concept maps 

Revised concept 
maps 


1st 

2nd 

3rd 

2nd 

3rd 


Structural 

Mean 

Structural 

Median 

Relational 

Mean 

Relational 

Median 

92.42 

93.5 

79.33 

84.5 

89.58 

88.0 

83.08 

86.0 

98.00 

97.5 

86.00 

80.5 

94.83 

89.5 

91.33 

97.0 

94.92 

92.5 

85.83 

89.0 
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The research aimed to clarify whether differences existed between students’ 
understandings of spatial concepts at the beginning, middle, and end of two 
introductory-level GIS courses. To answer this research question, the author analyzed 
differences between the scores of concept maps that were created and revised in three 
experiment sessions by conducting the Friedman’s ANOVA test and the follow-up 
Wilcoxon signed-rank test. The Friedman’s ANOVA test is a non-parametric test and 
for testing differences between two or more experimental conditions to which the same 
participants contribute. The Wilcoxon signed-rank test is a non-parametric post hoc test 
for the Friedman’s ANOVA test. The first analysis was the Friedman’s ANOVA test. 
For the concept map scored with the structural method, the scores did not change 
significantly over a semester (Table 10). For the concept map scores with the relational 
method (Table 11), the scores for concept maps created originally in the first session 
and the concept maps revised in the second and third sessions changed significantly 
(X 2 (2) = 6.68, p = 0.04). The Wilcoxon signed-rank test was next used to confirm 
whether results of the analysis of scores evaluated by the relational were significantly 
different (Table 12). A Bonferroni correction was applied, and so all effects were 
reported at a 0.0167 level of significance. It appeared that scores changed significantly 
in a comparison between the maps created in the first session (Mdn = 84.5) and the 
maps revised in the second session (Mdn = 97.0), Z = -2.51, p = 0.012. 

Table 10 


Result of the Friedman’s ANOVA Test of the Structural Scores 


Combination of Three Groups 

X 2 

P 

Created in session 1 - Created in session 2 - Created in session 3 

1.50 

0.47 

Created in session 1 - Created in session 2 - Revised in session 3 

3.36 

0.19 

Created in session 1 - Revised in session 2 - Created in session 3 

1.32 

0.52 

Created in session 1 - Revised in session 2 - Revised in session 3 

0.30 

0.86 

Table 11 

Result of the Friedman’s ANOVA Test of the Relational Scores 

Combination of Three Groups 

X 2 

P 

Created in session 1 - Created in session 2 - Created in session 3 

3.50 

0.17 

Created in session 1 - Created in session 2 - Revised in session 3 

4.87 

0.09 

Created in session 1 - Revised in session 2 - Created in session 3 

3.17 

0.21 

Created in session 1 - Revised in session 2 - Revised in session 3 

6.68 

0.04 

Table 12 

Result of the Wilcoxon Signed-Rank Test of the Relational Scores 

Pair of Two Groups 

Z 

P 

Created in session 1 - Revised in session 2 

-2.51 

0.012 

Created in session 1 - Revised in session 3 

-1.65 

0.099 

Revised in session 2 - Revised in session 3 

-1.51 

0.130 
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In an extended analysis of the first research question, the author statistically analyzed 
differences between participants enrolled in GEOG 390 and FRSC 461. This statistical 
analysis examined if there was any significant difference in the two courses regarding 
concept map score increase/decrease. To conduct this analysis, the author calculated the 
differences of two concept map scores by using the following five equations: (1) the 
score of the map created in the second session - the score of the map created in the first 
session; (2) the score of the map created in the third session - the score of the map 
revised in the first session; (3) the score of the map created in the third session - the 
score of the map created in the second session; (4) the score of the map revised in the 
second session - the score of the map originally created in the first session; and (5) the 
score of the map revised in the third session - the map originally developed in the 
second session. The results of the Mann-Whitney test indicated that there was not any 
significant difference between the two groups in either of the structural and the 
relational scores. 


Discussion 

The statistical analysis of the structural scores suggests that the structural scores did 
not change significantly over the course of the semester. A possible interpretation is that 
the structural scoring scheme could not detect the development of the participants’ 
conceptual knowledge structure. Participants arranged spatial concepts assigned by the 
researcher even though they were advised to use only the concepts with which they 
were familiar. If they were allowed to use freely the concepts that came to their minds, 
the scores might have had more variation and included more weighted map components 
such as crosslinks. This may have enabled the structural scoring scheme to detect 
participants’ knowledge structure development. Another possible interpretation is that a 
single semester is too short a period to detect the development of participants’ 
conceptual knowledge structures. The development of cognitive structures may 
gradually progress over a longer period than a single semester, or radical structural 
changes did not occur in a single semester. These possible factors may have hindered 
the improvement of map scores assessed by the structural. Since a radical or 
fundamental change in conceptual knowledge is a complex cognitive phenomenon, 
examining the process of the change may require different approaches (Rusanen, 2014). 

On the other hand, the statistical analysis of the relational scores showed that there 
was a statistically significant difference in score between the maps created in the first 
session and the maps revised in the second session (See Table 12). Students could 
improve their first concept maps at the middle of a semester. Thus, it could be said that 
the development of participants’ conceptual knowledge mainly occurred between the 
first and the second experiment sessions. A possible reason is that participants mainly 
learned concepts in the first half of the semester when both introductory-level GIS 
courses focused on fundamental geospatial concepts. In the latter half of the semester, 
the instructors mainly emphasized geospatial concepts applied to the spatial analysis. 
The relational scoring scheme is supposed to be sensitive to students’ understandings of 
concepts covered in instruction (Rice et al., 1998; Rye & Rubba, 2002). The relational 
scoring scheme may have detected the development of students’ conceptual knowledge, 
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suggesting that instructors use concept maps to assess students’ conceptual development 
in a time-series framework. 

Four avenues of future research could be fruitful. The first would be to repeat the 
work at hand using a larger number of participants. The small number of participants 
used for the current study were split across two introductory GIS classes, and may not 
have been sufficient to find modest but significant shifts in student’s understanding over 
the course of a single semester. A larger sample under full control of sampling would 
improve statistical rigor. The second would be to consider if and how instruction affects 
students’ geospatial concept learning. A researcher might conduct a multiple-group 
experiment including a control group and groups who receive special instruction to 
assess the effectiveness of teaching on concept learning. The third would be to consider 
if and how students’ spatial representations and reasoning improve as a consequence of 
the development of concept knowledge. For example, a researcher might compare 
students who appropriately understand the concept of map projection with students who 
do not understand in terms of their competency in spatial tasks that involve the concept. 
The fourth would be to clarify what kinds of geospatial concepts are complicated or 
simple for students in various circumstances. If researchers probe the complexity of 
geospatial concepts extensively in different settings, the results will identify which 
geospatial concepts should be taught before they teach a specific, more complicated 
geospatial concept. These four research themes would shed light on the improvement of 
spatial science curriculum and pedagogy. 

Concept mapping is a feasible way for a thorough understanding of concepts. 
Students analyze a topic, relate the relevant concepts, and reflect their understanding 
when they construct a concept map. Concept maps have potential in investigating how 
students develop their conceptual knowledge. This study adopted two scoring schemes: 
the structural and the relational. Most educators have assessed students’ concept maps 
by counting map elements and through quantitative methods. That being said, a 
quantitative method may evaluate a limited aspect of students’ learning. Evaluating 
statements shown in concept maps and adopting a qualitative method are viable 
approaches to assess the level of students’ development of conceptual knowledge 
(Draper, 2015; Miller et al. 2009). Use of a mix of quantitative and qualitative methods 
in this study suggested another course of action in evaluating students’ concept maps. A 
concept map is also a useful tool in teaching concepts. Instructors ask students to 
construct a concept map through collaborative work. This enables students to share 
knowledge and reach a consensus on the understanding of concepts through critical 
thinking (Draper, 2015; Schwendimann & Linn 2016). In another scenario of use of 
concept maps in education, pre-service teachers are recommended to learn how to use 
concept map for their instructions and assess students’ concept maps and how concept 
mapping impacts students’ learning (Subramaniam & Esprivalo Harrell, 2015). In such 
educational settings, evaluating students’ concept maps through the mixed methods 
would clarify the multiple aspects of students’ understanding of concepts. 
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Conclusion 

In this exploratory study, examining concept maps provided insights into if and how 
college students develop knowledge of geospatial concepts. Geospatial concepts are 
essential for spatial thinking through a wide variety of geospatial applications and 
technologies. These used to be taught exclusively through professional development for 
GIS experts but are now being taught by diverse disciplines and for learners of all ages 
(National Research Council, 2006; Sinton, Bednarz, Gersmehl, Kolvoord, & Uttal, 
2013). This allows everybody to have the opportunity to develop spatial thinking 
proficiency to become good decision makers in real-life situations (Kerski, 2008) and 
potentially obtain various jobs (Bednarz, 2004). Considering the significance and 
pervasiveness of geospatial technologies, it can be said that it is beneficial for educators 
to examine research and assessment methods for the development of students’ 
conceptual knowledge of geospatial sciences. GIS instructors should not assume that 
students automatically attain GIS conceptual knowledge. Rather, instructors should 
strategically introduce and reinforce geospatial concepts. 
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